Precision and Convergence of Monte Carlo Estimation of Two-Category UniODA Two-Tailed p

Monte Carlo (MC) research was used to study precision and convergence properties of MC methodology used to assess Type I error in exploratory (post hoc, or two-tailed) UniODA involving two balanced (equal N) classes. Study 1 ran 10 6 experiments for each N, and estimated cumulative p’s were compared with corresponding exact p for all known p values. Study 2 ran 10 5 experiments for each N, and observed the convergence of the estimated p’s. UniODA cumulative probabilities estimated using 10 5 experiments are only modestly less accurate than probabilities estimated using 10 6 experiments, and the maximum observed error (+0.002) is small. Study 3 ran 10 5 experiments for Ns ranging as high as 8,000 observations in order to examine asymptotic properties of optimal values for balanced designs.